TOJET: The Turkish Online Journal of Educational Technology-July 2012, volume 11 Issue 3 


PERCEPTIONS OF UNIVERSITY STUDENTS REGARDING COMPUTER 

ASSISTED ASSESSMENT 

Mubashrah Jamil 
mubashrahj @yahoo .com 


ABSTRACT 

Computer assisted assessment (CAA) is a common technique of assessment in higher educational institutions in 
Western countries, but a relatively new concept for students and teachers in Pakistan. It was therefore interesting 
to investigate students’ perceptions about CAA practices from different universities of Pakistan. Information was 
collected through a questionnaire, completed by 1877 students. It was concluded that overall sampled students 
were aware of CAA. Comparatively, female students were found to be more highly inclined towards CAA. 
Sampled students from different departments (i.e., Commerce, Mathematics and Pharmacy) and from 
universities D and E showed a particularly positive attitude towards CAA. However, overall sampled students 
felt that CAA overburdened them, and created some administrative and managerial problems, so teachers should 
probably avoid using this technique of assessment as routine. Prominent advantages regarding CAA referred to 
by the students included time saving, being more interesting, and being unbiased. Disadvantages included 
hardware and software related problems during examination. 

INTRODUCTION 

Learning is central to education (Desforges & Fox, 2002). The concept of learning may be thought as a 
‘consumer’ oriented term, while 'education' involves producers as well as consumers. Once learning is separated 
from education, it becomes incidental and could be only for fun (Jarvis, 2002). Assessment is an important factor 
that reflects and affects the quality of learning and education. Assessment plays a major role in driving student 
learning (Muwanga-Zake, 2006; Warburton, 2006; McLaren, 2008). According to the National Research Council 
(2003), ‘assessment is high quality information about students’ performance that informs teaching and learning’. 
Assessment identifies learner needs and can influence future planning if linked with appropriate resources. 
Moreover, assessment is a well organized and well defined measuring tool providing timely feedback for 
learning outcomes. Debuse, Lawley, and Shible (2008) highlighted the importance of feedback in assessment by 
characterizing it as informative, specific, personalized, timely, consistent, detailed and legible. 

Information and Communication Technologies (ICTs) offer many new possibilities for successful assessment. 
As ICTs were integrated in learning and assessment, educators demanded more effective, flexible, interactive, 
customized and just-in-time online instructional and assessment systems. Research exploring the role of ICTs in 
the teaching, learning and assessment process conclude that online learning and online assessment are impacting 
pedagogical developments in higher education (Hricko, Howell, 2005). Bach, Hynes and Smith (2007) and 
Warburton (2006) emphasized the importance of ICTs and explained that ‘universities have maintained the level 
of education for hundreds of years without the use of computer-based technologies, but the dominance of ICTs 
in the past two decades adds value to this process rather than frustrating it or even preventing it’. Due to the 
dominant impact of technology, the assessment process is gradually shifting from the traditional pen and paper 
method to computer based or e-assessment (Dube, Zhao, and Ma, 2009). E-assessment, online assessment, 
computer assisted or aided assessment; computerized assessment or computer based assessment is any 
assessment activity which involves the use of computers (Bull, 1999; Chalmers, McAusland, 2002; TAFE 
Frontier, 2002, Elliot, 2003). 

Pakistan is a country of 160 million people with only a 53% adult literacy rate. It relies on conventional means of 
imparting education within the limited resources of the educational budget, which is merely 2.1% of the GDP 
(Siddiqui, 2008); certainly not enough compared to international standards. Even with these limited resources, 
the Higher Education Commission (HEC) of Pakistan has been focusing on equipping higher education faculty 
with the latest ICT tools and techniques to increase the effectiveness of teaching and learning (HEC, 2010). It 
was personally observed by the researchers that all teachers from all departments of all public sector universities 
of Pakistan were assessing their students’ progress through paper and pencil based examination systems, except 
those departments who were offering ‘Computer Studies’ as a compulsory subject in their curriculum. Regarding 
this compulsory subject, the computer was partially being used by teachers and students during assessment. CAA 
is thus a relatively new term in the context of universities in Pakistan. Some teachers were using computer 
technologies in the process of assessment on the basis of their own interests and/or expertise, but computer 
technologies (i.e., construction, administration, marking and reporting directly on and via computer) were not 
fully implanted in any of the public sector universities of Pakistan. Therefore, it was important to know students’ 
perceptions about online examinations, whether they had experienced CAA before or not. 
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LITERATURE REVIEW 

The pressure for frequent assessment for the measurement of stated learning outcomes especially in those classes 
where large numbers of students are registered has resulted in the development of CAA. Bull (1999) focused on 
the impact of CAA on HEIs and proposed strategies for planning and implementation. She proposed that greater 
collaboration between supporting staff and academics could effectively reduce the implementation problems of 
online assessment. Bull concluded that CAA had the potential to impact institutions. Specifically, different 
format and types of assessment were available in CAA, and faster quality and speed of feedback could enhance 
the capability of academics to pinpoint student progress and learning deficiencies. Bull explained that if CAA is 
employed correctly, it has the power to ensure that curriculum modifications ensue and lead the institutions to 
revise existing assessment methods. The perceived validity of CAA and the risks associated with using 
technology were the major disadvantages highlighted by Bull. 

Chalmers and McAusland (2002) explained CAA in the context of pedagogical applications, noting that it 
enabled teachers to test their students covering a wide range of content, reduced teacher workload especially in 
the case of double marking, saved time and resources, and helped identify students’ learning problems by 
adapting to match their abilities. Administratively, CAA saves time in supervision, invigilation, marking and 
double marking, generating reports and communication with students. On the other hand, limitations of CAA 
were in terms of both pedagogical and administrative issues: 

• To construct objective questions needs adequate staff training, which is a time consuming process, 

• Frequent use of multiple-choice items can lead to a focus on testing a superficial level of understanding, 

• Additional training of IT staff is required for CAA invigilation, 

• To attempt an online examination, students should have a basic understanding of IT competencies, 

• An issue may rise regarding CAA for students with special needs, 

• Security measures are required for: student identification, preventing unauthorized access to question and 
answer files, cheating and plagiarism, 

• A high level of coordination is required among teaching staff, IT staff and administration. 

Moreover, Chalmers and McAusland conducted a case study on second year students of Economics which was 
aimed to understand the impact of CAA on their learning. They found that students’ understanding had 
improved. Their mean CAA score was higher than that from traditional essay-based examination. Students were 
encouraged by CAA to do some harder deep learning, which in the long run would be more valuable. 

TAFE Frontier (2002) designed an online survey to capture a snapshot of online assessment practices in 
vocational education and training, with a view to using the data to inform a guide to online assessment. A total of 
132 survey questionnaires were distributed, from which there were 43 responses via online, face to face or 
telephone interviews. Respondents defined online assessment as: An efficient and flexible means of conducting 
assessment and giving feedback; Using online technologies to support, wholly or in part, summative and 
formative assessment; Active participation by learners in an assessment event independently or as part of a group 
to achieve course outcomes; assessment where the assessor has not met the assessee; where participants have the 
opportunity to submit assessments through online channels, providing a more flexible approach to submitting 
work; demonstration of competency through the completion and submission of assessment tasks through the 
internet; a confused term and in most instances related to behaviorist views of student learning; Computer 
marked short quizzes to assess knowledge and online facilitator-marked assessments; to be able to determine the 
learner’s ability to understand the material using the internet; and assessment that involves some kind of 
electronic component, for example sending out, doing, submitting and collecting information from online 
resources. Online submission of assignments/essays; Multiple Choice Questions (MCQs), True/False and short 
answer based quizzes; email, web page design, problem solving, portfolios, journals and spreadsheets were 
reported as tools of online assessment. In a question about ‘suitability of online assessment to a range of study 
areas’, respondents argued that online assessment suited assessment of information retention and underpinning 
knowledge, rather than practical skills. One of them responded that video conferencing provide opportunities for 
properly supervised online assessment of practical skills. 

Sim, Holifield and Brown (2004) discussed issues related to the implementation of CAA, interoperability of 
questions, security, test construction and testing higher cognitive skills. Sim et al. explained that there are certain 
limitations in the different assessment methods of universities because of students’ different exposure to IT. 
They identified 50 various assessment techniques, with examinations and essays the most common among them. 
They defined CAA by quoting Bull and McKenna, who gave CAA as the most common term for the use of the 
computer in the assessment of students. They concluded that the most important lesson is that an institutional 
strategy greatly influences the success of the implementation of a CAA system. Therefore, it was recommended 
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to policy makers to formulate an effective strategy regarding CAA (e.g., security procedures, authentication and 
invigilation in remote locations). The second important lesson was concerned with staff development and 
training in test construction within a CAA environment. 

In the same direction, Conole and Warburton (2005) comprehensively reviewed the design, delivery and analysis 
of CAA. They explained that ‘computer-based assessment can be subdivided into stand-alone applications that 
only require a single computer, applications that work on private networks, and those that are designed to be 
delivered across networks such as the web’. They perceived CAA as high risk but central to the teaching and 
learning process. They also concluded that the role of technology and its expected impact on assessment was still 
in its infancy. It was required to develop new models for exploring this, and understanding more about the 
barriers and enablers in using these tools effectively. 

Muwanga-Zake (2006) evaluated the diagnostic value of CAA in the process of teaching and learning. They 
concluded that learners realized the usefulness of CAA and criterion-referencing as impacting successive 
learning perfonnance. Results showed that learners improved due to self-assessment and descriptive feedback 
which enabled them to diagnose and remedy mistakes themselves, through dialogue with or without assistance 
from the teachers. The free test atmosphere, instantaneous marking, and the diagnostic nature of test items 
supported self-diagnosis and led learners to request more testing, which ultimately increased competitiveness 
and excitement, especially when learners were aware of improvement. In case of teachers, they reflected that 
CAA could show learners’ problems and styles of thinking and also can test large number of students in short 
time. Moreover, CAA data could provide insight on teaching styles and problems. 

According to JISC (2008) ‘e-assessment is emerging as a major driver for e-leaming for both students and staff. 
They explained that on-demand testing facilitating the students having more control over their learning and 
moving toward more personalized learning. JISC added that in regard to questions about e-assessment, people 
have the following three assumptions: 

1. The assessment will be objective only, 

2. The assessment will be limited to multiple choice items only, 

3. The assessment can only test recall or low level learning outcomes. 

While pointing out the falsity of the above assumptions one-by-one, JISC explained that e-assessments has 
always involved subjective judgment and also reduces judgment bias. ICT has been incorporated into assessment 
in further and higher education in a number of ways, for example, the analysis of student postings to discussion 
boards in computer mediated conferencing and automated essay marking. In rejecting assumption 3, they noted 
that carefully constructed MCQs can assess higher order skills, and it was also possible to create questions, 
which might be easy to a student who has comprehended the topic whereas one who has not would struggle. 
JISC has also raised further issues associated with e-assessment: 

1. Time required: E-assessment saves time in displaying results including error checks and results analysis. 
But when the question is to assess the skills, the time and expertise required cannot be underestimated. 

2. Misleading clues: frequent quizzes from a particular area of content can mislead the students to guess 
what is and isn’t important. 

3. Equity and diversity: these are major issues when computers are involved in assessment process for 
different student groups relating to language, gender, computer anxiety and exam equivalence. 

4. Equivalence: the issues of equivalence between different forms of assessment (i.e., paper-pencil based 
compared to e-assessment) can affect student performance. Innovations in CAA introduced a new range 
of variants on this theme. 

5. Effects of using a full range of marks: assessment in mathematics and other science subjects which use a 
full range of marks may lend themselves to CAA. 

6. Greater Scrutiny: CAA has produced quality and efficiency gains in assessment, but even then there have 
sometimes been lower pass rates as compared to essay type assessment. 

7. Design of questions: to construct an MCQ-based test for e-assessment requires a higher order of skills and 
expertise in teachers. 

8. Disparity: the use of CAA has a positive impact on the test scores of high attaining pupils, when 
compared to those from paper-based tests, because higher-attaining students more quickly adapt to new 
assessment approaches. Lower-performing individuals will be disadvantaged when CAA is used, because 
a greater workload and additional effort is required to complete a computer-based test. 

9. Change in working practices: the time saving advantage of CAA has changed the working practices for 
academics in terms of acquiring training for designing and construction of e-assessments. Students are 
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also required to train for undertaking CAA, which raise the issue of who should be involved in that 
training and when it should take place, especially in the context of overloaded curricula. 

10. Plagiarism: CAA discourages plagiarism. A high level of imagination and rigor is required for the design 
of assessment online, compared to that for more traditional forms of assessment. 

11. Off-campus assessment: CAA enables the presentation of questions to students in different orders, with 
distracters in different orders, and if sufficient questions have been compiled of sufficient integrity, then 
they can be used for different tests. All of this allows for students to sit in adjoining desks in computer 
laboratories that will at other times be used for learning activities. This is fairly straightforward for on- 
campus students, but could be more problematic for students taking courses at a distance. However, 
Rovai (2000) suggests that this difficulty can be overcome by using 'proctored testing', where academics 
arrange for students to sit online assessments under test conditions in alternative venues. 

Debuse, Lawley and Shibl (2008) evaluated educators’ perceptions of automated assessment in terms of 
workload and quality of feedback. They used an automated feedback generator across multiple tutors on 
assessment items within an MBA course. They concluded that while the workload impact was generally positive 
with saving in both cost and time, improvements and modifications to the system could further reduce 
workloads. Specifically, improvements related to initial installation and training would significantly reduce the 
time required to be able to start using the system. The results indicated that AFG improved timeliness, 
consistency between markers, and recycling of useful comments. 

Similarly, other studies related to CAA were based on strategies for planning and implementation, models for 
designing CAA, and the role of CAA in formative, summative and diagnostic assessment. But very few studies 
were found on the topic of perceptions about CAA of students. One example was Sheader, Gouldsborough and 
Grady (2006), who analyzed the perceptions of staff and students by comparing CAA with traditional methods of 
assessment in practical Physiology classes. In an experiment, they utilized computer-based short-answer 
assessments for nearly 300 students and compared it to paper-based assessment of the same students. They 
concluded that students were generally well-disposed toward CAA (75% of the students responded that they had 
no preference in assessment methods in the future). Remote access to questions and ease of submission were 
positive aspects, but lack of internet access was a perceived disadvantage. On the other hand, the most common 
advantages cited by staff were reduction in marking time and paperwork, the improved quality of feedback to 
students, as well as the potential for the software to detect plagiarism and administer anonymous marking. The 
need to tailor questions to the technology and having to adapt answers for marking onscreen were the main 
disadvantages found by staff. The main difference between the Sheader et al/Debuse et al research studies and 
the present study is that CAA is not really applied in Pakistan. The above mentioned researches were conducted 
in situations where CAA was applied. Therefore this study is significant to all of those teachers, students and 
administrators who are interested in CAA in Pakistan. 

THE STUDY 

Sampling: In 2008 there were 111 (60 Public Sector and 53 Private Sector) universities in Pakistan (Higher 
Education Commission, 2008), out of which 36 (20 Public Sector and 16 Private Sector) universities from 
Punjab Province were delimited for this study. It was felt that this province would give a representative sample 
compared to other provinces, and it was convenient as the place in which the main researcher was located. Out of 
20 Public Sector Universities, 8 (40%) were randomly selected for inclusion in the sample. Private Sector 
Universities were dropped because of their limited number of students and the great variety of different 
disciplines they offered. All male and female students from all teaching departments of different disciplines of 
sampled universities constituted the population of this study. After the selection of the universities, different 
teaching departments of Pure Sciences, Social Sciences and Languages were randomly selected to be included in 
the study. However, students of each department could only be chosen if ‘available’ in their classes. Permission 
was sought from all of the head of the departments in each university before data collection. In the response of 
this request, 63% of them allowed to collect data from their students, while the remaining head of the 
departments were sent another request after 2.5 months to allow the researcher for data collection. At this time 
only 15% responded and rests were requested again and again after one month and then 15 days respectively. In 
all 2200 students were asked to complete the questionnaire. Of these, 1877 (85%) questionnaires were collected 
successfully after completion. 

Questionnaire : The questionnaire was based on questions developed for this study. Each question encouraged a 
response on a 5-point Likert Scale. All items of the questionnaire were couched in relation to issues emerging in 
the literature review. The methodology and format of the questionnaire followed the studies of Barbra and Sylvia 
(1999) and Yavuz (2005). It also stemmed from literature such as Debuse, Lawley and Shibl (2008), Irving, 
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Read, Hunt, and Knight (2000) and O’Hare (2001), who used a 5-point Likert scale model to measure student 
attitude towards CAA in different experimental studies conducted in different disciplines. 

The questionnaire was comprised of two major parts. Part 1 comprised of 12 items was generally for all 
students, while Part 2 consisted of 17 items and was for students who had some experience of online assessment. 
Part 2 was extended by adding two open ended questions regarding advantages and limitations of CAA. In Part 
1, generally simple items were included, but in Part 2 items were included which were directly related to their 
online assessment experience. 

To explore the issue of validity, the questionnaire was used in two pilot studies. In pilot study 1, the 
questionnaire was administered to students in three departments (Education, Mathematics and English) of two 
different universities (A and D). Sixty students including five males and five females in each of six departments 
completed the questionnaire. During administration of the questionnaire, any difficulties in understanding the 
terms, required space for answering the items and other questions raised by the students were recorded and then 
thoroughly discussed with a panel of experts. It was then decided to change the format by replacing some items, 
and include new items to make the questionnaire more self-explanatory. Due to the changes incorporated, it was 
then necessary to conduct a further pilot study. Departments of Business & Administrations were included in 
pilot study 2. Again, sixty students were chosen with equal distribution across departments and of gender. Minor 
revision was made as a result of the second pilot study. The final draft of the questionnaire was sent to the six 
different experts in the field of Education and Assessment for their opinion on. These six experts were 
Head/Director of departments in different universities of Punjab and were knowledgeable in the field of 
Assessment, Examinations and Computer Assisted Assessment. 

Response Rate and Analysis: The questionnaire was administered to 2200 students, of which 1877 (85%) 
responded successfully. Of these 1877 students, 1055 (56%) were female and 822 (44%) were male. Percentages 
and Pearson Chi Square (% 2 ) tests at a = 0.05 were applied to draw conclusions from the collected data. All 
positive statements in Parts 1 and 2 were weighted from +2 to -2 (Hirsch, Frank, Shapiro, Hazell and Frank, 
2004; Desselle, 2005) to analyze students’ response and negative statements in vice versa. Where +2 and +1 
rated ‘Strongly Agreed’ and ‘Agreed’ respectively while -1 and -2 were used for ‘Disagreed’ and ‘Strongly 
Disagreed’ respectively for all positive statements. And ‘0’ was meant by ‘Unable to Decide’. All responses to 
the open ended questions were analyzed in terms of themes emerging, and the frequency of the most commonly 
emerging themes was indicated in the form of percentages. 

FINDINGS 

It was found that 1052 (52% female, 62% male) students out of 1877 (56% female, 44% male) sampled students 
had had experiences of computer-based assessment. The examples they gave of these experiences were Army 
Medical College Entry Tests (2%); PAF (Pakistan Air Force) entry test (8%); online test or papers conducted by 
their teachers in universities (20%); computer practical in all computer-based subjects (60%); online NTS 
(National Testing Service), GAT (Graduate Assessment Test) online, GRE (Graduate Record Examinations) 
online examination (7%), Microsoft Online Certification Examinations (2%) and online TOEFL (Test Of 
English as a Foreign Language) online (1%). The data of both parts were interpreted in terms of whether student 
responses were an overwhelming majority (81% and above), a good majority (71% to 80%), or a simple majority 
(61% to 70%). 

Results from Part 1: Part 1 of the questionnaire was completed by all sampled students i.e., 1877. An 
overwhelming majority (i.e., more than 80%) of the sampled students reflected a positive attitude towards 
computer-based system of examinations in statements 2, 7, 11 and 12 (i.e., ‘students’ should be trained for 
computer-based system of examinations’, ‘CAA may help to improve and maintain standards of our examination 
system’, ‘CAA is requirement of the time’ and ‘CAA should be implanted in all universities’ respectively). A 
good majority (70%-80%) of the sampled students agreed that ‘CAA is an interesting technique of assessment’, 
‘interactive exercises may help to improve marks’, and ‘students should be given an opportunity to submit their 
assignment online’ (i.e., statements 1, 3, and 8 respectively). However, a simple majority of the sampled students 
agreed that ‘the computer does not need to be used in tests or exams’ (i.e., statement 4). 

The positive attitude of sampled students was counted per statement. It was found that all sampled students from 
universities A and E responded positively to eight different statements. Moreover, all sampled students from 
Biology, Commerce, Education and Mathematics departments responded positively to seven different 
statements. The positive attitude of sampled male students was counted per statement. It was found that sampled 
male students from university E responded positively in seven different statements. Eight positive statements 
were counted in departments of Chemistry. The positive attitude of sampled female students was counted per 
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statement. Sampled female students from universities A, E and H responded positively to eight different 
statements. Moreover, female students from departments of Commerce responded positively to eight different 
statements. 

Calculated values of % 2 showed significantly varying attitude differences between male and female students in 
statements 1, 2, 5, 6, 9, and 10 (Table 1). From the percentages, 79% male students in statement 1 were more 
interested in CAA than female (78%), while female students in statement 2 (85%), 5 (78%), and 9 (81%) were 
more positive towards CAA than to male (81%, 73%, 79% respectively) students. In statements 4 and 7 both 
male and female students were not clear about their decisions because of the minimum difference in the 
percentages of their responses. Calculated values of showed significantly varying attitude differences between 
sampled students from different universities in all statements except statement 7 (Table 1). Comparatively, 90% 
students from University E were more positive towards CAA. Calculated values of / 2 showed significantly 
varying attitude differences between the students from different departments in all statements (Table 1). It was 
observed that 97% of students from departments of mathematics agreed or strongly agreed that CAA saved time. 

Results from Part 2: It was found that 56% (1052) sampled students were having experience of online 
examinations. An overwhelming majority of the students reflected their positive attitude towards CAA, as ‘CAA 
is an interesting technique of assessment', they believe that it 'could support students to maintain their standard in 
examinations'; 'immediate feedback saves students’ time', 'it is reliable because results are always unbiased in 
online system of examination'; and 'cheating is almost impossible in online system of examination therefore it is 
much more secure technique of assessment’ (statements 1, 8, 10, 13). The same patterns of results were observed 
with respect to the students of all universities and of all departments. 

A good majority of the students declared their positive attitude towards CAA: ‘interactive exercises help to 
improve marks' and 'CAA is helpful in self assessment’ (statements 7 & 9). The same patterns of results were 
found by the students of all universities and all departments except Economics. However, a good majority of the 
students argued that ‘online system of examination overburdens the students to prepare for exams, so teachers 
should avoid it’ and that' uncertain and unexpected failure of technology during online examinations damages 
students’ (statement 15 and 16). The same patterns of results were found in sampled students from all 
universities except G and all departments except Biology and Mathematics. 

It was found that students of universities D and H and from departments of Pharmacy responded positively to six 
different statements out of 16 given in Part 2 of students’ questionnaire. Male students of universities C, D and H 
responded positively in five different statements and Business & Administration and Chemistry departments in 
seven out of 16 statements given in Part 2 of the questionnaire. It was found that female students of D and H 
universities responded positively and Biology, Commerce, CS & IT (Computer Science and Information 
Technology) and Pharmacy in seven different statements out of 16 statements. 

Calculated values of % 2 showed significantly varying attitude differences between all male and female students in 
statements 2, 7, 9, 11 and 16 (Table 2). It was found from the percentages that female students showed a 
comparatively more positive attitude towards CAA in all of these statements (i.e., 86%, 78%, 52%, 81% and 
92%) than male (i.e., 81%, 73%, 48%, 79%, and 90% respectively). Significantly varying attitude differences 
were found between the students from different universities in all statements except statements 8, 9, and 11 
(Table 2). It was found that 92% students from University D showed a more positive attitude towards CAA 
(statement 9). Significantly varying attitude differences were found between the students from different 
departments in all statements (Table 2). It was found that 100% students from departments of Psychology and 
Pharmacy showed a more positive attitude towards CAA (statements 13 and 14 respectively). 

The values of percentages regarding advantages and disadvantages, expressed by the experienced students were: 
CAA could save their time (n=277, 16%); female students were happy to find immediate feedback (n=79, 6%); 
male students agreed that online assessment is an interesting technique (n=57, 7%); unbiased judgments were 
available (n=56, 7%); and self-assessment could improve their GPA (n=56, 7%). On the other hand, female and 
male students estimated that it could be difficult to implement online examination systems (n=177, 11%) in 
universities. Female students faced some hardware and software problems (n=62, 7%) during their examinations 
and expressed that they saw their fellows in trouble because they were not experienced in online examination 
systems or not trained for using the internet (n=62, 7%). Male students also faced some troubles related to 
hardware and software problems during their exams (n=68, 9%). 
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CONCLUSIONS 

The research presented in this study investigated students’ perceptions of the contemporary utilization of CAA or 
online examinations in public sector universities of Punjab, Pakistan. IT experts and educationists in Pakistan are 
working hard to implement and improve computer assisted instruction (CAI) systems, but CAA or online 
examination methods are still ignored by them. This research has contributed by highlighting the importance of 
CAA to administrators, educationists and IT experts, who need to plan and implement computer technology in 
this phase of education. This research has also increased awareness about CAA among students and helped the 
researcher to plan and implement online examination systems in the near future. 

It was interesting to find that students were knowledgeable about online examinations, due to online admission 
tests of medical colleges; online NTS, GAT, GRE, MS Certification, TOEFL tests and computer-based 
classroom tests. They felt that online examination was not an exceptional technique of examinations for them. It 
was observed by the researcher during the survey that in all public sector universities a subject based on 
computer studies was compulsory for all students of all departments. Teachers assessed their students’ skills of 
computer applications, in that particular subject, online. 

From the results of Part 1 (completed by all students either having previously online examination experiences or 
not) it was clear that overall students had a positive attitude towards CAA. However, students from universities 
A and E and departments of Biology, Commerce, Education and Mathematics were comparatively more inclined 
towards CAA. Statistically significant attitude differences between the students of universities and different 
departments also reflected that the students from university E and departments of Mathematics were more 
interested in CAA. 

Male students from University E and departments of Chemistry were more positive towards CAA than to other 
males from other universities and departments. Similarly, female students from universities A, E and H and from 
departments of Commerce were found to be more interested in online systems of examinations than other 
females of other universities and departments. By comparing the values obtained from the above findings it was 
found that overall female students were more interesting in online systems of examinations than male students. It 
was confirmed from the results of % 2 test showing statistically significant differences between the attitude of 
male and female students. 

From the results of Part 2 (i.e., completed by only those students who had previous experiences of online 
examination), students were highly interested in CAA. Students from universities D and H and departments of 
Pharmacy were more inclined towards CAA. Statistically significant differences between the attitudes of 
experienced students from different universities and different departments showed that students from university 
D and departments of Pharmacy were more positive towards CAA. 

It was interesting to find that a simple majority (in Part 1) felt that ‘the computer does not need to be used in tests 
or exams’. Additionally, a good majority of students in Part 2 stated that ‘online systems of examination 
overburden students to prepare for exams’, so teachers should avoid conducting this type of examination 
routinely. Furthermore, due to the uncertain and unexpected failure of technology during online examination, 
damage to students’ could ensue, and ‘CAA is not adequate to apply in Pakistan’. 

Here, it is important to identify the two major administrative reasons of students’ negative attitude towards CAA. 
Firstly: problems of load shedding (i.e., breakage in power supply) and secondly: lack of technical staff and 
insufficient number of computer systems in computer laboratories. Due to these major reasons, students were 
losing their confidence in online systems of examinations and believed that ‘technology is not reliable’. In these 
situations, students preferred to take paper-based exams and tests. Researchers should help teachers and 
administrators to overcome these difficulties and make the online system of examinations interesting and 
valuable to all (students, teachers and academic staff). For example, it is clear from the literature that CAA is not 
an individual activity but requires collaboration between academic staff, supporting staff and administration 
(Bull, 1999). Bull further explained that CAA required a university-wide strategic approach and first piloting on 
small scale, enabling evaluation of the process and the development of the appropriate strategies to implement on 
a larger scale. 

The most prominent advantage observed during the survey was ‘immediate feedback saves student time’. In 
paper-based examinations students generally wait for a month (and some time more in the semester system) and 
more than 2 months (in the annual system) for their results. The most prominent disadvantage or limitation was 
the ‘impossibility of computer-based exams implementations in universities due to lack of resources and 
electricity failure problems.’ 
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It was concluded that CAA is an interesting technique of assessment. However, technological problems made 
this system of examination unreliable. If administrators of institutions are interested to apply CAA properly, then 
it will be important to take preliminary steps to control unexpected technological problems. If they are successful 
in controlling these problems, all teachers, students and administrators will find CAA a most beneficial system 
of examination. 
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APPENDIX 


Table 1 - Calculated Values of / 2 for Part 1 





Male & 

Between 

Between 

s. 

Statements 

x 2 

Female 

different 

different 

No. 

(a = 95%) 

(df=4, 

9.49) 

Universities 
(df=28, 41.34) 

Departments 
(df=44, 60.49) 

1. 

CAA is an interesting technique 

7 

T 

11.89* 

74.03* 

169.79* 

examination. 

Sig. 

0.18 

0.000 

0.000 

2. 

Students should be trained to use 

2 

1 

12.65* 

92.23* 

114.19* 

computer for online tests/exams 

Sig- 

0.013 

0.000 

0.000 


Interactive exercises help the students to 

2 

X 

8.16 

74.79* 

94.94* 

J . 

improve marks. 

Sig- 

0.086 

0.000 

0.000 

A 

Computer does not need to be used in 

2 

X 

0.56 

58.27* 

98.39* 


tests or exams. 

Sig- 

0.968 

0.001 

0.000 


If choice is given to me to select one 

2 

X 

13.56* 

59.73* 

101.13* 

5. 

from online testing or traditional 
examination system; I will prefer the 
second one. 

Sig. 

0.009 

0.000 

0.000 

6. 

I believe that CAA is not adequate in 

2 

X 

9.76* 

54.01* 

79.95* 

Pakistan. 

Sig. 

0.045 

0.002 

0.001 

7. 

CAA may help to maintain standard of 

X 2 

4.438 

39.61* 

121.84* 

examination at international level. 

Sig. 

0.350 

0.072 

0.000 

8. 

Students should be given an opportunity 
to submit their assignments online. 

2 

X 

Sig. 

1.99 

0.736 

50.33* 

0.006 

122.51* 

0.000 

9. 

CAA is a useless technique for me due 

2 

X 

14.63* 

59.95* 

111.13* 

to inadequate computer accessibility. 

Sig. 

0.005 

0.000 

0.000 

10. 

CAA is a useless technique for me 

X 2 

17.14* 

393.14* 

441.92* 

because I have no knowledge about it. 

Sig. 

0.002 

0.000 

0.000 

11. 

CAA is the requirement of the time. 

2 

X" 

Sig. 

8.09 

0.088 

187.38* 

0.000 

201.20* 

0.000 

12. 

CAA should be implanted in all 

2 

X 

6.29 

220.11* 

279.56* 

universities of Pakistan. 

Sig. 

0.178 

0.000 

0.000 


(* - significant values of % 2 ) 
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Table 2 - Calculated Values of/ 2 for Part 2 


s. 

No. 


r 

Male 

Female 

(df=4, 

9.49) 

Between 

Between 

Statements 

(a = 
95%) 

Universities 
(df=28, 41.34) 

Departments 
(df=44, 60.49) 

1 . 

CAA is an interesting technique 

x 2 

4.71 

57.58* 

174.61* 

examination. 

Sig. 

0.319 

0.001 

0.000 


Online tests/exams always made me 

X 2 

25.99* 

70.04* 

126.38* 

2. 

confused. 

Sig. 

0.000 

0.000 

0.000 


CAA can improve students’ 

X 2 

4.25 

55.69* 

86.36* 

J . 

achievements. 

Sig. 

0.373 

0.001 

0.000 


Minimum level of computer knowledge 

X 2 

5.61 

67.68* 

126.98* 

4. 

is enough to attempt the paper online. 

Sig. 

0.23 

0.000 

0.000 

5. 

CAA reduces tests/exam anxiety. 

X 2 

Sig. 

5.94 

0.204 

82.05* 

0.000 

115.79* 

0.000 

£ 

I believe that CAA is not adequate in 

X 2 

6.86 

59.69* 

74.71* 


Pakistan. 

Sig. 

0.143 

0.000 

0.003 


Interactive exercises help the students to 

X 2 

11.30* 

46.89* 

115.68* 

7. 

improve the test/examination marks. 

Sig. 

0.023 

0.014 

0.000 


CAA may help us to maintain our 

X 2 

5.15 

37.39 

100.67* 

8. 

standard of examination at international 
level. 

Sig. 

0.272 

0.110 

0.000 

9. 

CAA helps me to assess my-self. 

X 2 

Sig. 

11.68* 

0.020 

38.61 

0.087 

64.62* 

0.023 

10. 

Immediate feed back in online 

X 2 

2.287 

66.09* 

115.18* 

examination saves our time. 

Sig. 

0.683 

0.000 

0.000 

11. 

I always lose my confidence while 

X 2 

16.72* 

39.99 

92.55* 

solving the questions online. 

Sig. 

0.002 

0.066 

0.000 

12. 

I always prefer to take tests or exams 

X 2 

8.25 

45.08* 

98.38* 

online. 

Sig. 

0.083 

0.022 

0.000 


In online tests/exams, results are always 

X 2 

0.45 

157.77* 

118.55* 

13. 

unbiased. 

Sig. 

0.978 

0.000 

0.000 


Online tests/exams are much more 

x 2 

4.16 

214.79* 

202.07* 

14. 

secure than paper-based examination 
system. 

Sig. 

0.384 

0.000 

0.000 


Online examination technique 

X 2 

1.77 

151.49* 

176.87* 

15. 

overburdens the students, so it should be 
avoided by the teachers. 

Sig. 

0.779 

0.000 

0.000 

16. 

CAA damages thinking ability of 

X 2 

14.67* 

243.19* 

308.58* 

student during test/exam time. 

Sig. 

0.005 

0.000 

0.000 


(* - significant values of x ) 
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